Researchers have made notable progress in applying Large Language Models (LLMs) to solve math problems, as demonstrated through efforts like GSM8k, ProofNet, AlphaGeometry, and MathOdyssey. This progress has sparked interest in their potential use for tutoring students in mathematics. However, the reliability of LLMs in tutoring contexts -- where correctness and instructional quality are crucial -- remains underexplored. Moreover, LLM problem-solving capabilities may not necessarily translate into effective tutoring support for students. In this work, we present two novel approaches to evaluate the correctness and quality of LLMs in math tutoring contexts. The first approach uses an intelligent tutoring system for college algebra as a testbed to assess LLM problem-solving capabilities. We generate benchmark problems using the tutor, prompt a diverse set of LLMs to solve them, and compare the solutions to those generated by the tutor. The second approach evaluates LLM as tutors rather than problem solvers. We employ human evaluators, who act as students seeking tutoring support from each LLM. We then assess the quality and correctness of the support provided by the LLMs via a qualitative coding process. We applied these methods to evaluate several ChatGPT models, including 3.5 Turbo, 4, 4o, o1-mini, and o1-preview. Our findings show that when used as problem solvers, LLMs generate correct final answers for 85.5% of the college algebra problems tested. When employed interactively as tutors, 90% of LLM dialogues show high-quality instructional support; however, many contain errors -- only 56.6% are entirely correct. We conclude that, despite their potential, LLMs are not yet suitable as intelligent tutors for math without human oversight or additional mechanisms to ensure correctness and quality.

Intelligent tutors have proven to be effective in K-12 education, though their impact on adult learners -- especially as a supplementary resource -- remains underexplored. Understanding how adults voluntarily engage with educational technologies can inform the design of tools that support skill re-learning and enhancement. More critically, it helps determine whether tutoring systems, which are typically built for K-12 learners, can also support adult populations. This study examines the adoption, usage patterns, and effectiveness of a novel tutoring system, Apprentice Tutors, among adult learners at a state technical college. We analyze three types of data including, user demographics, grades, and tutor interactions, to assess whether voluntary tutor usage translates into measurable learning gains. Our findings reveal key temporal patterns in tutor engagement and provide evidence of learning within tutors, as determined through skill improvement in knowledge components across tutors. We also found evidence that this learning transferred outside the tutor, as observed through higher course assessment scores following tutor usage. These results suggest that intelligent tutors are a viable tool for adult learners, warranting further research into their long-term impact on this population.

This paper aims to uncover needs of adult learners when using pedagogical technologies such as intelligent tutoring systems. Further, our aim with this work is to understand the usability challenges when deploying tutors at scale within the adult learning audience. As educational technologies become more ubiquitous within k-12 education, this paper aims to bridge the gap in understanding on how adult users might utilize intelligent tutors. In pursuit of this, we built four intelligent tutors, and deployed them to 110 classrooms at a state technical college for an entire academic year. Following this deployment, we conducted focus groups amongst users to gather data to understand how learners perceived the optional educational technology during their academic journey. We further analyzed this data using foundational HCI methodologies to extract leanings and design recommendations on how developers might craft educational technologies for adoption at scale for the adult learning population.

In the past decade, we have observed a growing interest in using technologies such as artificial intelligence (AI), machine learning, and chatbots to provide assistance to language learners, especially in second language learning. By using AI and natural language processing (NLP) and chatbots, we can create an intelligent self-learning environment that goes beyond multiple-choice questions and/or fill in the blank exercises. In addition, NLP allows for learning to be adaptive in that it offers more than an indication that an error has occurred. It also provides a description of the error, uses linguistic analysis to isolate the source of the error, and then suggests additional drills to achieve optimal individualized learning outcomes. In this paper, we present our approach for developing an Artificial Intelligence-based Arabic Language and Speech Tutor (AI-ALST) for teaching the Moroccan Arabic dialect. The AI-ALST system is an intelligent tutor that provides analysis and assessment of students learning the Moroccan dialect at University of Arizona (UA). The AI-ALST provides a self-learned environment to practice each lesson for pronunciation training. In this paper, we present our initial experimental evaluation of the AI-ALST that is based on MFCC (Mel frequency cepstrum coefficient) feature extraction, bidirectional LSTM (Long Short-Term Memory), attention mechanism, and a cost-based strategy for dealing with class-imbalance learning. We evaluated our tutor on the word pronunciation of lesson 1 of the Moroccan Arabic dialect class. The experimental results show that the AI-ALST can effectively and successfully detect pronunciation errors and evaluate its performance by using F_1-score, accuracy, precision, and recall.

Machine learning algorithms enhanced medical students' technical performance and learning outcomes during a simulated brain tumor surgery, a new study shows. The COVID-19 pandemic has presented both challenges and opportunities for medical training. Remote learning technology has become increasingly important in several fields. The new study finds that in a remote environment, an artificial intelligence (AI) tutoring system can outperform expert human instructors. The Neurosurgical Simulation and Artificial Intelligence Learning Centre at The Neuro at Montreal Neurological Institute-Hospital recruited 70 medical students to perform virtual brain tumor removals on a neurosurgical simulator.

You are free to share this article under the Attribution 4.0 International license. Machine learning algorithms enhanced medical students' technical performance and learning outcomes during a simulated brain tumor surgery, a new study shows. The COVID-19 pandemic has presented both challenges and opportunities for medical training. Remote learning technology has become increasingly important in several fields. The new study finds that in a remote environment, an artificial intelligence (AI) tutoring system can outperform expert human instructors.

Machine learning algorithms enhanced technical performance and learning outcomes during simulated brain tumor removal. The COVID-19 pandemic has presented both challenges and opportunities for medical training. Remote learning technology has become increasingly important in several fields. A new study finds that in a remote environment, an artificial intelligence (AI) tutoring system can outperform expert human instructors. The Neurosurgical Simulation and Artificial Intelligence Learning Centre at The Neuro (Montreal Neurological Institute-Hospital) recruited seventy medical students to perform virtual brain tumor removals on a neurosurgical simulator.

The COVID-19 pandemic has presented both challenges and opportunities for medical training. Remote learning technology has become increasingly important in several fields. A new study finds that in a remote environment, an artificial intelligence (AI) tutoring system can outperform expert human instructors. The Neurosurgical Simulation and Artificial Intelligence Learning Center at The Neuro (Montreal Neurological Institute-Hospital) recruited seventy medical students to perform virtual brain tumor removals on a neurosurgical simulator. Students were randomly assigned to receive instruction and feedback by either an AI tutor or a remote expert instructor, with a third control group receiving no instruction.

Using a new method that employs artificial intelligence, a teacher can teach the computer by demonstrating several ways to solve problems in a topic, such as multicolumn addition, and correcting the computer if it responds incorrectly. Notably, the computer system learns to not only solve the problems in the ways it was taught, but also to generalize to solve all other problems in the topic, and do so in ways that might differ from those of the teacher, said Daniel Weitekamp III, a Ph.D. student in CMU's Human-Computer Interaction Institute (HCII). "A student might learn one way to do a problem and that would be sufficient," Weitekamp explained. "But a tutoring system needs to learn every kind of way to solve a problem." It needs to learn how to teach problem solving, not just how to solve problems.

Intelligent tutoring systems have been shown to be effective in helping to teach certain subjects, such as algebra or grammar, but creating these computerized systems is difficult and laborious. Now, researchers at Carnegie Mellon University have shown they can rapidly build them by, in effect, teaching the computer to teach. Using a new method that employs artificial intelligence, a teacher can teach the computer by demonstrating several ways to solve problems in a topic, such as multicolumn addition, and correcting the computer if it responds incorrectly. Notably, the computer system learns to not only solve the problems in the ways it was taught, but also to generalize to solve all other problems in the topic, and do so in ways that might differ from those of the teacher, said Daniel Weitekamp III, a Ph.D. student in CMU's Human-Computer Interaction Institute (HCII). "A student might learn one way to do a problem and that would be sufficient," Weitekamp explained.